WHAT IS CLAIMED IS: 



1 • A computer-implemented process for segmenting image data, 
comprising the process actions of: 
inputting an image; 

segmenting said image using a mean shift segmentation technique 
employing anisotropic kernels. 



2. The computer-implemented process of Claim 1 wherein the 

10 anisotropic kernels comprise a spatial/lattice component and a space dependent 
range/color domain component. 

3. The computer-implemented process of Claim 1 wherein the 
anisotropic kernels comprise a spatial/lattice component and a range/color 

15 domain component that is not space dependent. 



4. The computer-implemented process of Claim 1 wherein 
segmenting said image comprises: 
initializing kernel data; 

for each of a set of feature points, determining an anisotropic kernel with a 
spatial component and a related color component; 

associating a mean shift point with every feature point and initializing said 
mean shift point to coincide with that feature point; 

updating mean shift points by iterative anisotropic mean shift updates; and 
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merging vectors associated with feature points that are approximately the 
same to produce homogeneous color regions. 

5. The computer-implemented process of Claim 4 wherein initializing 
the kernel data comprises the process actions of: 

transferring pixels of said image into multi-dimensional feature points, 
specifying an initial spatial domain parameter h* and an initial range 
domain parameter % ; 

associating kernels with said feature points; 

initializing means of kernels as the value of said feature points associated 
with kernels; and 

setting initial kernel bandwidth matrices in the spatial/lattice domain as the 
diagonal matrix Hf = (A 0 5 ) 2 / and in the range/color domain setting h r (H?) = /i 0 r , 
where / is the identity matrix. 

6. The computer-implemented process of Claim 4, wherein for each of 
a set of feature points, determining an anisotropic kernel with a spatial/lattice 
component and a related range/color component, comprising the process actions 
of: 

for each feature point x„ searching the neighbors of said feature point x Jf 
j=l,...,n to obtain all feature points that satisfy the constraints of the kernels; 

iteratively updating a bandwidth matrix of the anisotropic kernel in the 
spatial domain, 
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and 



modulating the bandwidth of the anisotropic kernel in the spatial domain; 
modulating the color tolerance of the related color component. 



7. The computer-implemented process of Claim 6 wherein the 
constraints of kernels are defined by: 

k\ g (( Xi , Xj ,H^)<l^^^<l 

where H? is the spatial/lattice bandwidth matrix and h r is the range/color 
bandwidth parameter. 



8. The computer-implemented process of Claim 6 wherein the 
bandwidth matrix Hf is updated as: 
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9. The computer-implemented process of Claim 8 wherein the 
range/color bandwidth parameter h r (Hf ) is updated as: 

h'{H?)^^.h'{H?) 
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10. The computer-implemented process of Claim 6 wherein the 
segmentation is a single image segmentation and wherein modulations are 
applied to exaggerate eccentricity and modify scale. 

5 11. The computer-implemented process of Claim 6 wherein the 

segmentation is video segmentation and wherein modulations are applied for 
exaggerating eccentricity, scaling static segments, and overall scale. 

12. The computer-implemented process of Claim 4, wherein updating 
10 the mean shift points by iterative anisotropic mean shift updates, comprises the 

process actions of: 

for each mean shift point M (x, ) , 

determining the neighboring feature points x i ; 

calculating a mean shift vector M (*.) summing over all the 

15 neighboring mean shift points; and 

updating the mean shift points; 
until the change in the mean shift points is less than a specified amount. 

13. The computer-implemented process of Claim 12 wherein the mean 
20 shift vector is calculated as: 
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14. A system for segmenting image data, comprising: 

defining an anisotropic kernel of influence for each pixel in an image, 
wherein said kernel defines a measure of intuitive distance between pixels, 
where distance encompasses both spatial/lattice and range/color distance; and 

assigning to each pixel a mean shift point initialized to coincide with said 

pixel; 

iteratively moving each mean shift point upwards along the gradient of the 
kernel density function defined by the sum of all the kernels until they reach a 
stationary point; and 

considering pixels that are associated with the set of mean shift points that 
migrate to the approximately same stationary point to be members of a single 
segment. 

1 5. The system of Claim 14, further comprising: 
combining neighboring segments. 

1 6. The system of Claim 14, further comprising eliminating segments 
that contain less than a specified number of pixels. 

1 7. The system of Claim 14 wherein the image is a portion of video 
data and wherein distance further comprises temporal distance. 
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18. A computer-readable medium having computer executable 
instructions for segmenting image data, said computer executable instructions 
comprising: 

inputting image data; and 

segmenting said image data using a mean shift segmentation 
technique employing generally elliptical kernels wherein the computer-executable 
instruction for segmenting said image data comprises sub-instructions for: 
initializing kernel data; 

for each feature point, determining a kernel being a product of kernels with 
at least one of these kernels being elliptical; 

associating a mean shift point with every feature point and initializing said 
mean shift point to coincide with that feature point; 

updating mean shift points by an iterative anisotropic mean shift update; 

and 

merging vectors associated with feature points that are approximately the 
same to produce homogeneous color regions. 

1 9. The computer-readable medium of Claim 1 8 wherein an elliptical 
kernel comprises a spatial component.. 

20. The computer-readable medium of Claim 1 8 wherein a non- 
elliptical kernel comprises a color domain component.. 
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21 . The computer-readable medium of Claim 1 8 wherein the computer- 
executable instruction for segmenting said image data using a mean shift 
segmentation technique, comprises a sub-instruction for defining the shape of a 
elliptical kernel as XDAD T where X defines the overall volume of the kernel, A 
defines the relative lengths of the axes, and D is a rotation matrix that orients the 
kernel in space and time. 

22. The computer-readable medium of Claim 21 wherein the computer- 
executable instruction for segmenting said image data using a mean shift 
segmentation technique, further comprises a sub-instruction to modify the shape 
of an elliptical kernel by varying X , A or D . 

23. The computer-readable medium of Claim 1 9 wherein the image is a 
portion of video data and wherein the generally elliptical kernel further comprises 
a time component. 

24. The computer-readable medium of Claim 22 wherein by varying X 
the spatial size of the kernel is adjusted. 

25. The computer-readable medium of Claim 22 wherein by varying A 
the shape of the kernel is varied. 
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26. The computer-readable medium of Claim 25 wherein segmentation 
to segment elongated regions is encouraged by defining A as a diagonal matrix 
of Eigen values which is normalized to satisfy: 



1=1 

where a,, is the /'* diagonal elements of A , and a, > aj , for / < j ; and wherein the 



a, a : <=\ 



smaller Eigen values of A are diminished by: a t = \ ' ' ,/ = 2,...,p . 

[Jo; a t >\ 



27. The computer-readable medium of Claim 25 wherein larger 
segments for static objects are created by 
computing a scale factor s, as 

s,=a + (l-a)f[4(0 2 

where d x is the first Eigen vector in D , which corresponds with the largest Eigen 
value a, . </,(/) stands for the ith element in rf, , which is the x, y and t component 
of the vector when i = 1,2,3 , respectively, and a is a constant between 0 and 1 ; 

setting a to 0.25;. 

changing A to a] =a r s„i = 2,...,p; 



modifying A as a. t = 



P -\ 

s, =a+(l-a)Y[ d x (if \ and 



a]' 2 a : <=\ 



,— ,i = 2,...,p or modifying A as 



changing global scalar A as 
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28. The computer-readable medium of Claim 1 9 wherein said spatial 
kernel has a constant profile, k'(z) = 1 if | z |<l , and 0 otherwise. 

5 29. The computer-readable medium of Claim 20 wherein said color 

component uses an Epanechnikov kernel with a profile k r (z) = 1- 1 z | if | z |< 1 and 
0 otherwise. 
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